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DETAILED ACTION 



1 . This action is responsive to communications of application received 12/12/2003. 

2. The disposition of the claims is as follows: claims 1 - 7 are pending in the application. 
Claim 1 is the only independent claim. 

3. The group and/or Art Unit location of your application has changed. To aid in the 
correlation of any papers for this application, all further correspondence should be directed to 
Group Art Unit 2624 (effective 03/07). Please be sure to use the most current art unit number on 
all correspondence to help us route your case and respond to you in a timely fashion. 

4. When making claim amendments, the applicant is encouraged to consider the references 
in their entireties, including those portions that have not been cited by the examiner and their 
equivalents as they may most broadly and appropriately apply to any particular anticipated claim 
amendments. 



Specification 

5. The disclosure is objected to because of the following informalities: On page 5 of the 
specification, reference is given to Figure 2 A in second line of paragraph [018]. Reference is also 
made to Figures 2A-2B in fourth line of paragraph [020]. Yet the there are no Figures 2 A nor 2B 
labeled in the drawings. 

Appropriate correction is required. 
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Claim Rejections - 35 USC § 103 

6. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

7. Claims 1 and 2 are rejected under 35 U.S.C. 103(a) as being unpatentable over Shibata et 
al., (US 20020061 136 Al), hereinafter Shibata, and further in view of Choi, (US 20020042793 
Al). 

A. Shibata discloses claim 1, "A method for learning a structure of a video to detect events 
in the video consistent with the structure, comprising: 

selecting sets of features from the video [para. 0017 and 0109, corresponding to feature 
amounts suitable to select a set of feature amounts]; 

updating a hierarchical statistical model for each set of features [para. 0031, Fig. 1 is a 
schematic view showing a hierarchical model of video data; and para. 0052, see Fig. 1]; 

evaluating an information gain of the hierarchical statistical model [para. 0104, 
corresponding to 'extracts a statistic representative value of an entire segment ... n-dimensional 
vector ... histogram and a power spectrum are involved; para. 0105 -0106]; 

filtering redundant features [para. 0101 corresponds to 'determines the extraction number 
of features amounts by balancing maximization of the fidelity and minimization of the data 
redundancy in the segment representation']; 

updating the hierarchical statistical model based on the filtered features [para. 0104 - 
0107, corresponding to 'method of forming a dissimilarity measurement criterion for a feature 
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amount representative of an extracted dynamic feature is based on the dissimilarity measurement 
criterion for the static feature amount on which the dynamic feature amount is based.']; 
applying a Bayesian information criteria to each model and feature set pair; and 
rank ordering the model and feature set pairs to learn the structure and detect the events 
in the video in an unsupervised manner" supra [as detailed]. 

However, Shibata does not appear to disclose "applying a Bayesian information 
criteria to each model and feature set pair [Choi: Abstract, para. 0002 - 0005, 0023 - 0027, 0032, 
0034, 0051, 0079, 0099, 0173, 0196 - 0197, corresponding to 'Bayesian self-organizing feature 
maps (SOM)', and 'Bayesian statistical technique']; and 

rank ordering the model and feature set pairs to learn the structure and detect the events in the 
video in an unsupervised manner [Choi: Abstract, para. 0002 - 0005, 0023 - 0027, 0032, 0034, 
0051, 0079, 0099, 0173, 0196 - 0197, corresponding to 'order-rank document clusters ... is a 
type of an unsupervised learning']", but Choi does [as detailed]. 

Therefore it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to apply 'feature amounts, hierarchical model of video data, statistic 
representative value of an entire segment and minimization of the data redundancy disclosed by 
Shibata in combination with 'Bayesian self-organizing feature maps (SOM)', and 'Bayesian 
statistical technique' disclosed by Choi, and motivated to combine the teachings because Shibata 
is directed toward audio visual signal processing of video data, and although Choi is directed 
toward performing real-time document clustering for relevant documents in accordance with a 
degree of semantic similarity , Choi would improve the accuracy of information retrieval via 
performing real-time clustering for relevant information as revealed in para. 0002. 
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B. Shibata and Choi disclose claim 2, "The method of claim 1 , in which the hierarchical 
statistical model uses Gaussian mixtures" supra for claim 1 and furthermore by Choi in para. 
0029 corresponding to 'Gaussian distribution'. 

Therefore it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to apply 'feature amounts, hierarchical model of video data, statistic 
representative value of an entire segment and minimization of the data redundancy disclosed by 
Shibata in combination with 'Bayesian self-organizing feature maps (SOM)', 'Bayesian 
statistical technique' and 'Gaussian distribution' disclosed by Choi, and motivated to combine \ 
the teachings because Shibata is directed toward audio visual signal processing of video data, 
and although Choi is directed toward performing real-time document clustering for relevant 
documents in accordance with a degree of semantic similarity , Choi would improve the 
accuracy of information retrieval via performing real-time clustering for relevant information as 
revealed in para. 0002. 

(Examiner's note: Also see IDS reference: "Discovering Recurrent Events In Video Using 
Unsupervised Methods", section (3.2 Movies Videos), 'Each state emits observations using 
Gaussian mixture model (GMM)'.) 

8. Claims 3 and 4 are rejected under 35 U.S.C. 103(a) as being unpatentable over Shibata et 
al., (US 20020061 136 Al), hereinafter Shibata, in view of Choi, (US 20020042793 Al), and 
further in view of Lin et al., (US 7,076,102 B2), hereinafter Lin, or Ozer et al., (US 
2004/0120581 Al), hereinafter Ozer, and/or Sterz, Walter et al., (DE 10122212 CI), hereinafter 
Sterz. 
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A. Shibata and Choi disclose claim 3, "The method of claim 1, in which the hierarchical 
statistical model uses hidden Markov models" supra for claim 1 . However they do not appear to 
disclose "in which the hierarchical statistical model uses hidden Markov models". But Lin does 
in col. 1, Ins. 64-67, at Tor example, probabilistic techniques, such as Hidden Markov models 
(HMMs) and Bayesian networks, have been used extensively to recognize complex motion 
patterns and to learn and recognize human activities' or 

Ozer does in para. [0036] and [0066] corresponding to 'Hidden Markov Models' 
and/or Sterz does in front page corresponding to 'Hidden Markov Models'. 
Therefore it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to apply 'feature amounts, hierarchical model of video data, statistic 
representative value of an entire segment and minimization of the data redundancy disclosed by 
Shibata in combination with 'Bayesian self-organizing feature maps (SOM)', and 'Bayesian 
statistical technique' disclosed by Choi, and motivated to combine the teachings because Shibata 
is directed toward audio visual signal processing of video data, and although Choi is directed 
toward performing real-time document clustering for relevant documents in accordance with a 
degree of semantic similarity , Choi would improve the accuracy of information retrieval via 
performing real-time clustering for relevant information as revealed in para. 0002., coupled with 
Hidden Markov Models disclosed by Lin, Ozer or Sterz and motivated to couple the teachings 
because they all similarly classified as related art according to Current US and US Cross 
Reference Classifications. 

B. Shibata, Choi, Lin or Ozer, and Sterz disclose claim 4, "The method of claim 3, in which 
states of events in the video are modeled as low-level hidden Markov models in the hierarchical 
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hidden Markov model, and the events are modeled as a high-level Markov chain in the 
hierarchical hidden Markov model" supra for claim 3, and furthermore by Lin in col. 1, Ins. 64- 
67 at 'For example, probabilistic techniques, such as Hidden Markov models (HMMs) and 
Bayesian networks, have been used extensively to recognize complex motion patterns and to 
learn and recognize human activities', wherein 'complex motion patterns' and 'human activities' 
correspond to "high-level Markov chains" (i.e. state transitions: golf video -> tee shot -> pan -> 
ball through air -> lands and rolls on fairway - specification para. 0003); or furthermore by Ozer 
in para. [0036] and [0066] at 'movements' (of body object parts) determined by 'Hidden Markov 
Models', wherein 'movements' (of body object parts) also corresponds to "high-level Markov 
chains" (i.e. state transitions: golf video -> tee shot -> pan -> ball through air -> lands and rolls 
on fairway - specification para. 0003); and 

furthermore by Sterz in front page at 'Training model for pattern recognition, especially 
Hidden-Markov model, involves defining displacement between related training patterns by 
correlating patterns so that correlation is maximal' wherein 'displacement between related 
training patterns' corresponds to "states of events in the video are modeled as low-level hidden 
Markov models" (i.e. color schemes, texture patterns, or motion - specification para. 0003). 

Therefore it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to apply 'feature amounts, hierarchical model of video data, statistic 
representative value of an entire segment and minimization of the data redundancy disclosed by 
Shibata in combination with 'Bayesian self-organizing feature maps (SOM)', and 'Bayesian 
statistical technique' disclosed by Choi, and motivated to combine the teachings because Shibata 
is directed toward audio visual signal processing of video data, and although Choi is directed 
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toward performing real-time document clustering for relevant documents in accordance with a 
degree of semantic similarity , Choi would improve the accuracy of information retrieval via 
performing real-time clustering for relevant information as revealed in para. 0002., coupled with 
Hidden Markov Models disclosed by Lin, Ozer or Sterz and motivated to couple the teachings 
because they all similarly classified as related art according to Current US and US Cross 
Reference Classifications. 

9. Claim 5 is rejected under 35 U.S.C. 103(a) as being unpatentable over Shibata et al., (US 
20020061 136 Al), hereinafter Shibata, in view of Choi, (US 20020042793 Al), and further in 
view of Official notice. 

A. Shibata and Choi disclose claim 5, "The method of claim 1 , in which the features include 
dominant color ratios, motion intensity, a least-square estimates of camera translation, audio 
volume, spectral roll-off, low-band energy, high-band energy, zero-crossing rate (ZCR)" supra 
for claim 1 . However, they do not appear to disclose, "in which the features include dominant 
color ratios, motion intensity, a least-square estimates of camera translation, audio volume, 
spectral roll-off, low-band energy, high-band energy, zero-crossing rate (ZCR)", but Official 
notice is taken that the art is replete with features that include dominant color ratios, motion 
intensity, a least-square estimates of camera translation, audio volume, spectral roll-off, low- 
band energy, high-band energy, zero-crossing rate (ZCR). 

Therefore it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to apply 'feature amounts, hierarchical model of video data, statistic 
representative value of an entire segment and minimization of the data redundancy disclosed by 
Shibata in combination with 'Bayesian self-organizing feature maps (SOM)', and 'Bayesian 
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statistical technique' disclosed by Choi, and motivated to combine the teachings because Shibata 
is directed toward audio visual signal processing of video data, and although Choi is directed 
toward performing real-time document clustering for relevant documents in accordance with a 
degree of semantic similarity , Choi would improve the accuracy of information retrieval via 
performing real-time clustering for relevant information as revealed in para. 0002, and further 
coupled with "features that include dominant color ratios, motion intensity, a least-square 
estimates of camera translation, audio volume, spectral roll-off, low-band energy, high-band 
energy, zero-crossing rate (ZCR)" disclosed by Official notice. 

For instance see IDS reference: "Structure Analysis of Soccer Video With Hidden Markov 
Models", first paragraph at 'dominant color ratio and motion intensity, based on the special 
syntax and content characteristics of soccer videos'. 

10. Claim 6 is rejected under 35 U.S.C. 103(a) as being unpatentable over Shibata et al., (US 
20020061 136 Al), hereinafter Shibata, in view of Choi, (US 20020042793 Al), and further in 
view of Bremer et al., (US 2005/0176057 Al), hereinafter Bremer. 

A. Shibata and Choi disclose claim 6, "The method of claim 1, in which the features are 
filtered with a Markov blanket" supra for claim 1 . However, they do not appear to disclose, "in 
which the features are filtered with a Markov blanket", but Bremer does in para. [0028] at 
'Preferred feature selection algorithms include, but are not limited to, Forward or Backward 
Floating, SVMs, Markov Blankets, Tree Based Methods with node discarding, Genetic 
Algorithms, Regression-based methods, kernel-based methods, and filter -based methods.' 

Therefore it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to apply 'feature amounts, hierarchical model of video data, statistic 
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representative value of an entire segment and minimization of the data redundancy disclosed by 
Shibata in combination with 'Bayesian self-organizing feature maps (SOM)\ and 'Bayesian 
statistical technique' disclosed by Choi, and motivated to combine the teachings because Shibata 
is directed toward audio visual signal processing of video data, and although Choi is directed 
toward performing real-time document clustering for relevant documents in accordance with a 
degree of semantic similarity , Choi would improve the accuracy of information retrieval via 
performing real-time clustering for relevant information as revealed in para. 0002, and further 
coupled with Markov Blanket filter-based method disclosed by Bremer and motivated to couple 
the teachings because although the field of art seems unrelated it actually is related in the sense 
of feature selection as applied to a training set comprising patients and their associated values. 

1 1 . Claim 7 is rejected under 35 U.S.C. 103(a) as being unpatentable over Shibata et al., (US 
20020061 136 Al), hereinafter Shibata, in view of Choi, (US 20020042793 Al), and further in 
view of Altschuler et al., (US 6,012,052), hereinafter Altschuler. 

A. Shibata and Choi disclose claim 7, "The method of claim 1, in which the evaluating is 
performed using expectation maximization and a Markov chain Monte Carlo method" supra for 
claim 1 . However, they do not appear to disclose, "in which the evaluating is performed using 
expectation maximization and a Markov chain Monte Carlo method", but Altschuler does in col. 

12, Ins. 10-13 at 'Alternatively, the number of clusters specified may be averaged over or 
estimated using known statistical methods such as reversible jump Markov Chain Monte Carlo 
algorithms' and in col. 14, Ins. 57-61 and in col. 16, Ins. 10-19 at 'Expectation Maximization (or 
"EM") algorithm'. 
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Therefore it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to apply 'feature amounts, hierarchical model of video data, statistic 
representative value of an entire segment and minimization of the data redundancy disclosed by 
Shibata in combination with 'Bayesian self-organizing feature maps (SOM)', and 'Bayesian 
statistical technique' disclosed by Choi, and motivated to combine the teachings because Shibata 
is directed toward audio visual signal processing of video data, and although Choi is directed 
toward performing real-time document clustering for relevant documents in accordance with a 
degree of semantic similarity , Choi would improve the accuracy of information retrieval via 
performing real-time clustering for relevant information as revealed in para. 0002, and further 
coupled with 'Markov Chain Monte Carlo algorithms' and "Expectation Maximization (or 
"EM") algorithm' disclosed by Altschuler and motivated to couple the teachings because the 
methods to pre-fetch resources and build resource link topology templates may also be used for 
collaborative filtering as disclosed by Altschuler in col. 3, Ins. 60-67. 

Responses 

12. Responses to this action should be mailed to: Commissioner of Patents and Trademarks, 
Washington, D.C. 20231. 

Inquiries 

13. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Gregory F. Cunningham whose telephone number is (571) 272- 
7784. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Matt Bella can be reached on (571) 272-7778. The Central FAX Number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 




Gregory F. Cunningham 
Examiner, Art Unit 2624 
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